An Intelligent Marshalling Plan Using a New Reinforcement Learning System for Container Yard Terminals

In recent years, the number of shipping containers grows rapidly, and in many container yard terminals, increasing throughput of material handling operation becomes important issue as well as decreasing the turnaround times of vessels. Material handling operations for loading containers into a vessel is highly complex, and the complexity grows at an exponential rate according to the growth of the number of containers, the operation occupy a large part of the total run time of shipping at container terminals. A challenge of this chapter is focused on improving throughput of the material handling operations for loading container on a vessel by using reinforcement learning. Commonly, materials are packed into containers and each container in a vessel has its own position determined by the destination, weight, owner, and so on (Siberholz et al., 1991; Günther & Kim, 2005). Thus, containers have to be loaded into a ship in a certain desired order because they cannot be rearranged in the ship. Therefore, containers must be rearranged before loading if the initial layout is different from the desired layout. Containers carried into the terminal are stacked randomly in a certain area called bay and a set of bays are called yard. The rearrangement process conducted within a bay is called marshalling.